Skip to content

[gfx942][opus] Support A8W8 blockscale BpreShuffle GEMM - #4204

Merged
yifehuan merged 3 commits into
mainfrom
yifehuan/a8w8_opus_gemm
Jul 13, 2026
Merged

[gfx942][opus] Support A8W8 blockscale BpreShuffle GEMM#4204
yifehuan merged 3 commits into
mainfrom
yifehuan/a8w8_opus_gemm

Conversation

@yifehuan

Copy link
Copy Markdown
Contributor

Motivation

Add an optimized Opus backend for FP8 A8W8 blockscale B-preShuffle GEMM on gfx942.

The main goals are:

  • provide a gfx942 Opus implementation for the DeepSeek-V4 A8W8 blockscale GEMM shapes
  • integrate Opus into the existing A8W8 tuning and runtime dispatch flow
  • preserve existing CK/ASM/CKTile and gfx950 behavior
  • support fresh JIT builds and CI-style multi-architecture builds
  • maintain numerical accuracy and performance across ROCm versions

Technical Details

Key changes:

  • Added a gfx942 Opus A8W8 blockscale B-preShuffle kernel and associated traits.
  • Added gfx942 architecture definitions required by the new pipeline.
  • Extended Opus code generation to support the A8W8 kernel using the same generated host/device translation-unit structure as the existing Opus GEMM kernels.
  • Added an explicit kernel-ID tuning entry point:
    • opus_gemm_a8w8_blockscale_bpreshuffle_tune
    • current tuned kernel ID: 11000
  • Integrated the Opus backend into the common A8W8 blockscale tuner.
  • Added support for both legacy kernelId and newer solidx tuning-result columns.
  • Added 118 tuned gfx942 DeepSeek-V4 shapes to:
    • aiter/configs/model_configs/dsv4_a8w8_blockscale_bpreshuffle_tuned_gemm.csv
  • Kept the existing gfx950 tuned entries unchanged.
  • Fixed the fresh-JIT operator registration so the explicit kernelId argument is forwarded correctly.
  • Made generated device translation units define __HIPCC_RTC__ locally, allowing both normal JIT and prebuilt multi-architecture builds to use the lightweight device-TU host pass without requiring changes to setup.py.
  • Kept the A8W8-specific ROCm compiler configuration local to the Opus module.

Test Plan

Validation was performed with ROCm 7.0 and ROCm 7.2 containers.

Coverage included:

  • Black formatting checks for all Python files changed by this PR
  • Ruff checks for all Python files changed by this PR
  • fresh JIT compilation of module_aiter_core
  • fresh JIT compilation of module_deepgemm_opus
  • CI-style multi-architecture compilation with:
GPU_ARCHS="gfx942;gfx950"

- fatbin inspection to confirm both targets are present:
    - amdhsa--gfx942
    - amdhsa--gfx950

- correctness and performance validation for representative Opus shapes:
    - (M, N, K) = (512, 4096, 1024)
    - (M, N, K) = (1408, 1536, 4096)
    - (M, N, K) = (1920, 4096, 2048)
    - (M, N, K) = (7296, 8192, 1024)

- regression checks for existing CK A8W8 tuning paths
- verification that existing gfx950 tuned CSV entries remain unchanged

## Test Result

Formatting and lint:

- Black: passed
- Ruff: passed
- git diff --check: passed

Build validation:

- fresh ROCm 7.0 JIT build: passed
- fresh ROCm 7.2 JIT build: passed
- gfx942;gfx950 multi-architecture Opus build: passed
- generated fatbin contains both gfx942 and gfx950 code objects

Representative ROCm 7.2 multi-architecture results:

    M       N       K    Latency (us)    Error ratio
━━━━━━  ━━━━━━  ━━━━━━  ━━━━━━━━━━━━━━  ━━━━━━━━━━━━━
  512    4096    1024          29.995       0.006921
──────  ──────  ──────  ──────────────  ─────────────
 1408    1536    4096          76.035       0.001647
──────  ──────  ──────  ──────────────  ─────────────
 1920    4096    2048         120.078       0.006735
──────  ──────  ──────  ──────────────  ─────────────
 7296    8192    1024         502.826       0.007270

The multi-architecture results are consistent with the corresponding single-architecture fresh build. Existing gfx950 tuned shapes and dispatch behavior are unchanged.

## Submission Checklist

- [ ] Look over the contributing guidelines at https://github.com/ROCm/ROCm/blob/develop/CONTRIBUTING.md#pull-requests.

@yifehuan
yifehuan requested a review from a team July 12, 2026 12:33
@github-actions

Copy link
Copy Markdown
Contributor

🏷️ CI Guide

Runs automatically on every PR:

  • ✅ Pre-checks (submodule verification, code formatting)
  • ✅ Aiter op tests (gfx942 + gfx950)
  • ✅ Triton tests on MI35X (only when aiter/ops/triton/** or related paths are changed)

Extended tests (opt-in via labels):

Label Tests
ci:triton-300x Run an additional Triton test job on MI300X in PRs; main branch always runs both MI35X and MI300X
ci:sglang SGLang integration tests: DeepSeek-R1-MXFP4 accuracy, Qwen 3.5 accuracy
ci:atom ATOM benchmark: DeepSeek-R1-0528, GPT-OSS-120B
ci:atom_full ATOM accuracy suite for PR and main models from ATOM models_accuracy.json
ci:vllm vLLM benchmark: GPT-OSS-120B, DeepSeek-R1-0528, Kimi-K2.5
ci:all All standard extended tests (excludes ci:atom_full)

Only add ci:atom_full for FlyDSL or Triton upgrades.
Add labels via the sidebar or gh pr edit 4204 --add-label <label>

@yifehuan
yifehuan force-pushed the yifehuan/a8w8_opus_gemm branch from 2052234 to 93e17af Compare July 12, 2026 12:55
@zufayu
zufayu requested a review from yzhou103 July 13, 2026 01:37
@yifehuan
yifehuan merged commit cdf01e2 into main Jul 13, 2026
44 checks passed
@yifehuan
yifehuan deleted the yifehuan/a8w8_opus_gemm branch July 13, 2026 06:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants